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ABSTRACT 

This paper describes a highly successful peer tutoring program that has resulted in an 
improvement in the passing rates of mathematics placement exams from 16% to 42%, on average. 
Statistical analyses were conducted using a Chi-Squared (X 1 2 3 4 ) test for independence and the results 
were statistically significant (p-value much less than 0.05). These encouraging results suggest 
that this type of initiative should be institutionalized at the end of the grant period that funds it. 
The paper describes the structure of the program, the conditions of the university in which it is 
being implemented, the details of the statistical analyses, and its impact in engineering education. 
This model may easily be replicated in academic institutions with similar conditions. 
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INTRODUCTION 


he spark that ignited this effort was the realization that placement exams provide a unique opportunity 
for first-year students to quickly advance in the sequence of mathematics courses that is required of 
any engineering curricula. Tutoring is offered at all levels of Algebra (considered remedial) as well 
as for Pre-Calculus II (trigonometry) and Calculus I which are the first two engineering-level courses. The virtues 
of this program, which is offered in the summer previous to starting the first year of college, are the following: 



1. The student quickly progresses along the mathematics critical path. Each placement exam accelerates the 
potential graduation date by one semester. The student benefits by an earlier graduation date and the 
university benefits by improving the graduation rates in engineering. 

2. For students who require tutoring at the most remedial of levels, the placement exams solve a critical 
problem in course scheduling. Most engineering courses, as well as the natural sciences component, have 
Calculus I as a prerequisite. If two or three semesters are required to come up to speed, the student will 
deplete all the possible courses that they can register for. Their academic load reduces to a single 
mathematics course. This situation is not favorable for anyone. 

3. Many of the students that highly benefited from the summer mathematics tutoring program (SMTP) had 
already been exposed to part of the material covered during the tutoring sessions, even though it did not 
reflected as such in the mathematics scores of their college entry exam. Because of this, the students might 
lose interest during their first semester mathematics course since they are not placed correctly. 

4. The student saves money since the tutoring services and the placement exams are free for them. In 
addition, they do not have to pay the registration fee for the courses as they would if they registered for 
them during the semester. The SMPT serves as mitigation tool for the change in the paradigm of how to 
finance their college tuition as a result of the changes in Pell grant regulations. 
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5. It is offered during the summer when students have the time to immerse themselves in the SMPT and the 
placement exams. This provides an excellent opportunity for proactive students to empower themselves 
and take an active role in their immediate academic future. 

6. Students that participate in the SMPT do so out of their own free will. The fact that a student chooses to 
participate in this effort, instead of idling during the summer, creates a very positive attitude in the student. 
This attitude will serve the student very well once they start the rigor of the engineering curriculum. 

7. This program offers, in many cases, new engineering students their first exposure to a college-level 
instructional and educational setting, as it helps lay the initial framework needed to succeed in a demanding 
academic career. 

Recognizing the importance and potential benefits that the placement exams might have on these students’ 
academic career, a peer-tutoring program was developed. This approach was selected instead of a regular tutoring 
sessions or teacher-based class because the benefits of peer-tutoring throughout STEM (science, technology, 
engineering, and mathematics) careers are well documented, particularly in remedial courses (Bowen, et al., 2007; 
Newman-Ford, et al., 2007). From the literature review executed by Henderson, Fadali, & Johnson (2002), they 
concluded that peer tutoring has a positive effect on student achievement and retention. Also, from their own 
research, they observed that students generally show a positive attitude towards tutoring. A study developed by 
Magin & Churches (1995) concluded that the use of peer tutoring in engineering is an useful, viable teaching 
strategy for achieving course goals. Multiple case studies (Chen & Liu, 2011; Luca & Clarkson, 2002) have shown 
the positive impact that tutoring provides on the tutee performance. Also, Power & Dunphy (2010) describe the 
peer tutoring environment as a “safe and non-threatening interactive learning situation”. This favorable setting 
promotes student engagement which is critical in their first college year. Several studies point to the fact that 
engagement for first year college students is critical in successfully retaining them in STEM related fields (Knight, 
et al., 2003; Krause and Coates, 2008; Tinto, 2006). 

This sort of activity also exposes students to a different learning scenario while providing them with an 
opportunity in which their effort could be rewarded rather quickly, since they can pass one or more placement exams 
during summer. Students that are willing to invest time of their summer vacation into this type of activity are 
showing compromise towards their college career and a willingness to improve themselves. These experiences have 
been shown to be important aspects in the learning process (Farkas, 2003). Boud, Cohen & Sampson (2001) 
mention multiple benefits besides grade improvement that peer-tutoring programs provide, such as improvement of 
skills related to working with others and self-management skills utilized in the learning process. 

There is an additional advantage obtained from this teaching strategy, which is the deeper learning of 
concepts obtained by the tutors. Beasley (1997) and Davis & Wilcock (2004) point out the benefits that peer- 
tutoring sessions, whether they are formal or informal, provide for tutors themselves, making it a valuable 
experience for the tutor’s personal and professional development. When peer tutoring works, two students benefit 
simultaneously from the process. Therefore, student tutors can, in fact, learn from their teaching experiences 
(Roscoe & Chi, 2008). Atkinson et al. (2003) argue that the tutors’ necessity to use examples to demonstrate 
important concepts and processes improves the self-learning process, as research on learning from worked-out 
examples has shown that they gain a deeper understanding by dissecting examples into steps and associating them to 
fundamental principles. 

Therefore, the peer-tutoring approach for improving the student’s performance before taking placement 
exams seems appropriate. 

ABOUT THE UNIVERSITY 

Universidad del Turabo (UT) is part of the Ana G. Mendez University System (AGMUS), one of the 
largest private university systems in Puerto Rico. The vision statements of all the engineering programs {to become 
the first choice for all motivated students who wish to pursue an engineering education) stem from the ideals of the 
founder of AGMUS, the late Ms. Ana G. Mendez. She initiated the AGMUS educational system 65 years ago as a 
means for Puerto Rican youth who, although capable and motivated, lacked the necessary skills required to continue 
university-level studies. In many cases these students were, and still are, the first generation in their families to 
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attend a university. Ms. Ana G. Mendez opened the doors of opportunity to this segment of the population. The 
word “all” in the vision statement indicates that, while keeping the spirit of opportunity that inspired Ana G. 
Mendez, the engineering programs are not limited by it. The School of Engineering (SoE) strives to attract highly 
skilled students, which is done primarily through very competitive scholarships. The word “motivated” is also 
important as the students, once admitted, are put through the rigor of an engineering curriculum regardless of their 
background. Thus, although opportunity is provided to enter the UT SoE, only motivated students will be able to 
endure the curriculum. Realizing that the potential students that are interested in studying engineering will have 
variable backgrounds, the UT SoE decided to implement the SMTP described in this paper to provide better tools 
and opportunities to its freshmen. 

STRUCTURE OF THE TUTORING PROGRAM 

The SMTP began during the 2012 summer, as an initiative to tackle a long-standing situation. It started 
intertwined with the ongoing semester tutoring service in May 2012, during the last weeks of the semester. The 
semester tutoring program had tutors available from 9:00 am through 9:00 pm from Monday to Thursday and from 
9:00 am through 5:00 pm on Fridays, servicing the students of the School of Engineering (SoE). The SoE students 
were required to reach out to the tutors during their work schedule or through email for scheduling appointments. 
This methodology, although successfully implemented within the students of the SoE, proved to be a challenge for 
high school students. They were not ready to participate in this kind of process because they lacked the discipline 
and structure to study by themselves, bring specific doubts to the tutors, and to schedule appointments. Realizing 
this, efforts where focused to develop a tutoring program specific to work with the needs of this group. 

Two tutors, seniors in the SoE, were hired specifically for tutoring these students during the 2012 summer 
months. Because of time constraints and Human Resource regulations, only one tutor was supposed to be working 
at a time. Regardless of this, both tutors where present most of the time because of their willingness to help. One 
computer laboratory, with capacity for seating up to 30 students, was dedicated for the mathematics tutoring 
sessions. It was open from 9:00 am to 4:00 pm from Monday to Friday during six weeks of summer, and students 
were free to come during this period to better accommodate their schedules. The tutoring sessions utilized the 
Educosoft CJ online platform (also used in UT Mathematics courses), which was provided by the Mathematics 
Department, free of charge. Independent study sessions took place under the supervision and tutelage of the two 
tutors. Approximately, 8 students were assisted daily, with some days reaching up to 15 students simultaneously. 
Students usually visited the SMTP for various days before taking the placement exam. 

Halfway during the summer tutoring sessions, both tutors identified that most of the students were lacking 
important concepts that hindered their progression and that many of them shared the same challenges. This situation 
prompted the evolution of the tutoring program, as tutors started teaching short classes on the topics that students 
were having the most difficulties with. These classes were improvised on the spot and were not structured, but 
proved to be productive because most of the students took advantage of them. During the last two weeks of the 
SMTP, more than three mini classes were being taught on a daily basis, with most of the students that were present 
participating in them. 

In the closure meeting of the SMTP, the tutors themselves brought to our attention that most of the students 
that assisted to the tutoring sessions needed classes and remedial work, not just individual study sessions with 
support. They suggested that the next iteration of the program should include a more a course-like component to 
tackle most of the subjects that students were having problems with. Also, the evaluation forms that students filled 
out after the tutoring sessions pointed out their interests in a more traditional, class-like approach for the SMTP. 

During the next year, efforts were made to meet the challenge of developing an intensive remedial class 
that could be taught during one week of summer. This class should be able to help students achieve their potential 
in mathematics, which would result in them passing one or more placement exams before their first semester. One 
of the principal requirements for preparing the class was to model it based on the syllabi of the UT math courses. 
Therefore, with help of the UT Mathematics Department, the syllabi of the impacted courses (Introductory Algebra 
with laboratory. Introductory Algebra, Intermediate Algebra, College Algebra, and Trigonometry and Analytical 
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Geometry) were utilized for preparing the course. Also, available copies of old departmental exams were used to 
both determine the course content and as supplementary material. 

In order to provide a more comprehensive experience for the students, different computer tools were also 
researched. Some requisites considered for selecting the product were that the software was available to the students 
online and throughout the day. Also, the software had to have the capability of helping out students in solving 
problems without giving away the answers. It was essential for the project that students had access to a tool 
permanently so they could work on their own schedule and receive some level of support if needed during irregular 
study hours. 

The book utilized for preparing the course material was Matematicas universitarias introductorias con 
nivelador MyMathLab (Introductory College Mathematics with MyMathLab Leveler, by Demana and Blitzer, 
2009). This book was specially prepared to cover the essential mathematics topics needed for entry level college 
courses. Also, its content aligned well with UT mathematic courses requirements. The book was complemented 
with Pearson’s MyMathLab online platform, which was an excellent tool for students to practice and work on their 
skills. The MyMathLab platform was perfectly aligned with the chosen book and it also had an electronic version of 
the book available at all times for the students. 

During the 2013 summer, the improved tutoring sessions were conducted. For a total of 8 weeks, a 1-week 
long intensive class was taught to students that had enrolled earlier. The class was free of charge and used the 
materials acquired by the project. Students were required to enroll previously for the week-long course and since 
the course was intensive and time sensitive, they were not allowed to enter the course mid-week. The independent 
study group was maintained because some students stated that they were not interested in a week-long class and 
only wanted to refresh specific concepts. Four (4) tutors and a graduate student that was already working with the 
project were hired for the 2013 summer session. They were divided into two (2) groups, the one that handled the 
class-like tutoring sessions and the one that worked with the independent study group. To comply with Human 
Resources regulations, tutors where assigned either morning or afternoon shifts. It was an unexpected, yet 
remarkable outcome that the tutors, out of their free will and commitment to both the project and the students, 
decided to stay the whole day with the students. This fact was one of the most important aspects of the success of 
the program. 

The independent study group was maintained, but it quickly did not had as much impact as expected, since 
students gravitated towards the more familiar, class-like sessions. What quickly became evident to the tutors was 
that there were two distinct groups with different mathematic-level needs. Therefore, efforts were made to segregate 
the students according to their entry level mathematics course. One group was made of students that entered in both 
Introductory Algebra courses, while the other one grouped students that entered at a higher level (Trigonometry and 
Analytical Geometry). Also, the classes were adjusted to the separate levels to better fit the needs of the students in 
each group. The last weekly tutoring sessions were carried out following this structure and student feedback was 
very favorable with this approach. 

CONDITIONS OF THE TESTING FACILITIES: PAST AND PRESENT 

Placement exams are one of the tools utilized by UT to properly allocate new students in various 
disciplines, including mathematics. Since they are free and approving them results in receiving full credit for the 
corresponding course, they are an extremely powerful yet underutilized tool that entry-level students have. 
Placement exams were administered to students the same day that they visited the UT campus for admission. The 
mathematics placement exam is computer-based, and no resources besides pencils and papers are allowed. The 
students have to score 14 out of 20 questions correctly in order to pass an exam. It is noteworthy that this is a 
“Yes/No” proposition, meaning that the student has to solve perfectly each problem, as no partial credit is allowed. 
Because of lack of space, students usually take the placement exam in a back office that doubles as an examination 
center. The office partitions are not ceiling-high, meaning that environmental sounds from surrounding offices are 
heard in the examination site. Also, noises and distractions inherent from the traditional office work (phone, 
paperwork, etc.) are usually present at the time of examination. Realizing this, the Title V (Developing Hispanic - 
Serving Institutions Program of the United States Department of Education) STEM UT project administrative staff 
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coordinated with the Associate Vice-Chancellery for Retention to postpone the administration of the mathematics 
placement exam to SoE candidates. Also, the students are allowed whenever they pass an exam to leave and return 
later to take the next level exam, providing them with a chance to rest and prepare better for the next one. 

PROGRAM EVALUATION BY THE STUDENTS 

Participants of the SMTP were asked to fill out the same evaluation questionnaire that SoE students fill 
during their sessions. The 5-item questionnaire strives to capture the student feedback by using a 10-point Likert 
scale, where 10 and 0 were the highest and lowest scores possible, respectively. Table 1 presents the questions or 
statements presented to the 2012 summer students and their summarized answers. 


Table 1: Student Evaluation Of The SMTP 


Question 

Rate Ave. 

Std. Dev. 

1. How good do you consider the work of the tutor? 

9.74 

0.80 

2. How do you rate the resources of the UT? 

9.35 

1.18 

3. Please quantify your level of understanding of concepts prior consulting with the tutor: 

6.20 

2.36 

4. Please quantify your level of understanding of concepts prior after the tutoring session: 

8.89 

1.51 

5. Overall, how much you believe that the tutoring helped in your understanding of the concepts? 

9.47 

0.95 


It can be appreciated from the overall results that the perception of students regarding the SMTP is a very 
positive one. It is noteworthy that the answers to questions 1, 2, and 5, which measure the student satisfaction with 
the process are above nine (9) with small standard deviations, which means answers where consistent. Question 3, 
which asks for perception of knowledge before the tutoring session took place, had a 6.2 average value, with a 
considerably large standard deviation. The large standard deviation suggests that their perception on previous 
knowledge is varied, but the prevalent consensus is that they feel that they are not prepared for college level 
engineering mathematics. This particular statistic is very relevant as it confirms the importance of extending this 
service to these students and the possibility to institutionalize this program once project funding is over. 

RESULTS 

During the 2011 summer, no tutoring sessions where offered. A total of 191 students went to take the 
placement exam. Of those, 8 backed out of the exam just before it. 183 students took exams, of which 26 approved 
one or more exams. In terms of exams taken, 207 exams where taken, with only 31 exams approved. These data is 
used as a baseline before the summer tutoring sessions where introduced. 

During the 2012 summer, the first iteration of the summer SMTP took place. A total of 151 students went 
to take the placement exam, and of those, 36 students backed out of the exam just before taking it. 115 students took 
exams, for a total of 135 exams. 

• No tutoring: 135 out of the 151 students did not take tutoring sessions. Of the 36 students that did not take 
the exam, 31 did not take tutoring sessions. Of the 115 students that took the exam, 104 did not take 
tutoring sessions. Of those 104, 18 students passed 1 or more exams, for a total of 21 exams passed. 

• With tutoring: 16 out of the 151 students took tutoring sessions. Of the 36 students that did not take the 
exam, 5 took tutoring sessions. Of the 115 students that took the exam, 11 took tutoring sessions. Of those 
11,5 students passed 1 or more exams, for a total of 6 exams passed. 

During the 2013 summer, the renewed SMTP was implemented. A total of 135 students went to take the 
placement exam, and of those, 19 backed out of the exam just before taking it. 116 students took exams, for a total 
of 136 exams. 
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• No tutoring: 45 out of the 135 students did not take tutoring sessions. Of the 19 students that did not take the 
exam, 13 did not take tutoring sessions. Of the 116 students that took the exam, 32 did not take tutoring 
sessions. Of those 32, 5 students passed 1 or more exams, for a total of 5 exams passed. 

• With tutoring: 90 out of the 135 students took tutoring sessions. Of the 19 students that did not took the exam, 
6 took tutoring sessions Of the 116 students that took the exam, 84 took tutoring sessions. Of those 84, 36 
students passed 1 or more exams, for a total of 44 exams passed. 

STATISTICAL ANALYSES 

Statistical Analysis 1: 

In order to validate the effects that the program had over the passing rates of placement exams, a statistical 
analysis was performed. It was hypothesized that there should be a positive correlation between taking mathematics 
tutoring and passing one or more placement tests. Statistically significant results would provide evidence to 
conclude that the SMTP will continue to be effective in future years and that, if replicated elsewhere, the SMTP 
could lead to an improvement in mathematics placement exam results. Therefore, a Chi-Squared test for 
independence was performed to determine if there is a statistically significant association between tutoring and the 
passing rate in mathematics placement exams. As suggested by Greenwood (1996), Chi-Squared analyses are 
adaptable to testing multivariate distributions, thus providing an appropriate framework for analyzing the program’s 
effects. The objective of presenting a high level of detail in this section is to have the paper also serve as a tutorial 
for readers who are unfamiliar with a Chi-Squared test of independence. 

A Chi-Squared test compares two categorical variables. Categorical variables have a qualitative rather than 
quantitative nature; for example, male/female, pass/fail, cold/hot. The categorical variables may have more than two 
levels; for example, low/medium/high, North/South/East/West. Also, in some cases the qualitative variable may 
seem numerical but it is in reality qualitative in nature; for example, classifying values by year where each year is a 
number but serves as a qualifier. In addition, a numerical variable may be turned into a categorical variable by 
assigning ranges of values to particular levels; for example, the numerical values of a test may be turned into a 
pass/fail categorical variable. In fact, categorizing the mathematics placement exam results into pass/fail levels was 
done in this paper to perform the Chi-Squared test. 

In a Chi-Squared test the researchers must decide on which of the two variables plays the role of the 
explanatory variable. In this paper the explanatory variable is “Tutoring”, that is, whether the student participated in 
tutoring sessions or not. The other variable is known as the response variable. In this paper the response variable is 
the Exam Result divided into Pass/Fail categories. With this setup the results in the mathematics placement exam 
are explained by whether the student was tutored or not. The observed placement exam results for the years 2011- 
2013 are presented in Table 2 which is presented in the form required for a Chi-Squared test, that is, it includes the 
row and column sums. The table also includes the totals for the three-year period. 

Table 2: Observed Placement Exam Results 


2011 

Pass 

Fail 

Totals 

With Tutoring 

0 

0 

0 

No Tutoring 

31 

176 

207 

E 

31 

176 

207 


2013 

Pass 

Fail 

Totals 

With Tutoring 

44 

60 

104 

No Tutoring 

5 

27 

32 

E 

49 

87 

136 


2012 

Pass 

Fail 

Totals 

With Tutoring 

6 

9 

15 

No Tutoring 

21 

99 

120 

E 

27 

108 

135 


11-13 Totals 

Pass 

Fail 

Totals 

With Tutoring 

50 

69 

119 

No Tutoring 

57 

302 

359 

E 

107 

371 

478 


Table 3 summarizes the conditional percentages from Table 2, that is, the percentages of the response 
variable (passed or failed the placement exam) for each value of the explanatory variable (tutored/not tutored). The 
table shows that a larger percentage of students who attended the tutoring sessions passed one of the mathematics 
placement exams. For example, looking at the totals (bottom of Table 3), 42% of the students who attended the 


Copyright by author(s); CC-BY 


66 


The Clute Institute 































American Journal of Engineering Education - December 2014 


Volume 5, Number 2 


tutoring sessions passed a placement exam. In contrast, only 15.9% of the students who were not tutored passed the 
placement exam. Similar results are shown for each individual year (note that tutoring started in 2012 so that the 
2011 results only include students that were not tutored). 

The results in Table 3 provide evidence to indicate that the tutoring sessions have been very effective and 
that tutoring is strongly related to passing a placement exam; however, these results by themselves are not enough to 
conclude that the relationship would hold in a larger population of students. This issue will be investigated further 
by establishing a hypothesis test. The results of the hypothesis test will establish if the evidence is strong enough to 
conclude, beyond a reasonable doubt, that the improvement in passing rates must be related to the tutoring sessions. 
If the evidence is not strong enough, then the results could be attributed to chance, i.e., sample variability, and not 
necessarily because a relationship exists between the two variables. 


Table 3: Conditional Percentages Of Observed Counts 


2011 

Pass 

Fail 

Totals 

With Tutoring 

N/A 

N/A 

N/A 

No Tutoring 

31/207= 15% 

176/207= 85% 

207(100%) 


2012 

Pass 

Fail 

Totals 

With Tutoring 

6/15= 40% 

9/15= 60% 

15 (100%) 

No Tutoring 

21/120= 17.5% 

99/120= 82.5% 

120 (100%) 


2013 

Pass 

Fail 

Totals 

With Tutoring 

44/104= 42.3% 

60/104= 57.7% 

104 (100%) 

No Tutoring 

5/32= 15.6% 

27/32= 84.4% 

32 (100%) 


Totals 

Pass 

Fail 

Totals 

2011-13 

With Tutoring 

50/119= 42% 

69/119=58% 

119(100%) 

No Tutoring 

57/359= 15.9% 

302/359= 84.1% 

359 (100%) 


H 0 (Null Hypothesis): There is NO difference in placement exam passing rates between students who are tutored 
and those who are not, that is, the two variables are independent from each other. 

H a (Alternate Hypothesis): There is a difference in placement exam passing rates between students who are tutored 
and those who are not, that is, that there is a relationship between the two variables (they are not independent). 

The null hypothesis in a Chi Square analysis implies that there are an equal proportion of students who passed the 
placement exams to those who did not pass. 

To conduct the Chi Square test for independence it is necessary to calculate a new set of counts - the 
Expected Counts - which is based on the premise that the two variables are independent from each other, that is, 
assuming that the null hypothesis was true. The expected count essentially asks the following question: how many 
students would be expected to pass the exam if there was no relationship between tutoring and passing rates? 

Since the expected counts are based on the assumption that the two variables are independent, as 
established by the null hypothesis, then we may use the basic probability rule that states that “the probability that 
two events A and B will occur is the product of their individual probabilities, or P(A and B) = P(A)*P(B)”. By 
following this simple rule on each of the four cells of the table, the expected counts can be calculated. 

The following example illustrates the calculation for the top-left cell of the table of totals. For the top-left 
cell, A = “Passed the exam” and B = “with tutoring”. The calculation for the individual probability uses a row or a 
column sum and divides it by “n”, the number of examinees. 
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The computation of expected count for top-left cell of the table of totals is as follows: 

P(A and B) = P(Passed the Exam and was tutored) = P(Passed the Exam)*P(Tutored) 

P(B) = PIPassed the Exam) = 107/478 = 22.38% 

P(B) = P(Tutored) = 119/478 = 24.9% 

P(A and B) = (107/478)*(119/478) = 5.57% 

Since there were a total of 478 examinees, if passing the exam and tutoring were independent of each other, then the 
count of examinees passing the exam with tutoring would be expected to be: 

Examinees Total*P(Passed the Exam with Tutoring) = 478*(107/478)*(119/478) = 27 

The same procedure can be carried out for each of the remaining three cells: 

Examinees Total*P(Passed the Exam with no Tutoring) = 478*(107/478)(359/478) = 80 
Examinees Total*P(Failed the Exam with Tutoring) = 478*(371/478)(119/478) = 92 
Examinees Total*P(Failed the Exam with no Tutoring) = 478*(371/478)(359/478) = 279 

Check the sum of expected counts: 27 + 80 + 92 + 279 = 478 (ok) 


Totals 

Pass 

Fail 

Totals 

2011-13 

50 

69 

119 

With Tutoring 

57 

302 

359 

Z 

107 

371 

478 


The Chi Square test measures how far the expected counts are from the observed ones and the conclusion is 
based on the size of the discrepancy between them as formulated below: 


x 


2 


I 


all cells 


(Observed Count — Expected Count ) 2 
Expected Count 


Table 4: Observed And Expected Counts. Expected Counts In Parentheses 


Totals 

Pass 

Fail 

Totals 

2011-13 

50 (27) 

69 (92) 

119 

With Tutoring 

57 (80) 

302 (279) 

359 

Z 

107 

371 

478 


The discrepancy between observed and expected counts, as measured with the Chi-Squared statistic, is 
considered “large” (statistically significant) if it is greater than 3.84 for a 2x2 case. This corresponds to a 
significance level of 95%. The corresponding minimum p-value = 1.0 - 0.95 = 0.05. The smaller the p-value, the 
more likely it is that there is an association between the variables. 

For this case, the Chi Square statistic is 

X 2 = (50-27) 2 /27 + (57-80) 2 /80 + (69-92) 2 /92 + (302-279) 2 /279 
X 2 = 19.6 + 6.6 + 5.7 + 1.9 
X 2 = 33.8 

Since the value X 2 = 33.8 is much larger than the required cutoff X 2 = 3.84 for a 2x2 case, the conclusion is that the 
results are statistically significant. 

The problem has also been solved using the R programming language for more precise results. The X 2 
result (35.14) calculated by R is slightly higher than the result calculated manually (33.8). This is due to round-off 
errors during the manual calculation which rounded the “expected counts” to the nearest integer. R also computes 
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the p-value and it shows that it is smaller than 3.056xl0" 9 which is much lower than the minimum required (p- 
value) lllin = 0.05. 

The chi square test for independence clearly shows that the null hypothesis can be rejected, i.e., that the two 
variables (tutoring and passing rates) are not independent from each other. Therefore, the alternate hypothesis is 
accepted. It may be concluded that there is a statistically significant association between receiving tutoring and the 
passing rates in the placement exam. The results provide evidence that the improved passing rates in mathematics 
placement exams will continue if the tutoring sessions continue in the future or, if the tutoring sessions are replicated 
elsewhere (another university or program). 

Statistical Analysis 2: 

A second Chi-Squared test for independence was conducted to determine if there was any association 
between the year that the mathematics placement test was taken (explanatory variable) and the passing rate for 
students that were not tutored (response variable). If there is no association - which is the expectation - the result 
may be generalized into the past and the future (entire population), i.e., when there is no tutoring, the passing rates in 
the mathematics placement exams will be low irrespective of the year the exam is taken. 

Table 5 contains the observed counts, and the row and column sums. Table 6 contains the conditional 
percentages of the observed counts. Table 7 contains the expected counts. The procedure to obtain the percentages 
and the expected counts is the same one carried out previously. In this case, the matrix has 3 rows (three years) and 
2 rows (pass, fail). 


Tab le 5: Observed Counts Per Year (Only For Cases With No Tutoring) 



Pass 

Fail 

£ 

2011 (No tutoring) 

31 

176 

207 

2012 (No tutoring) 

21 

99 

120 

2013 (No tutoring) 

5 

27 

32 

Sum 

57 

302 

359 


Table 6: Conditional Percentages Of The Observed Counts (Only For Cases With No Tutoring) 



Pass 

Fail 

£ 

2011 (No tutoring) 

31/207 = 15% 

176/207=85% 

(207) 100% 

2012 (No tutoring) 

21/120 = 17.5% 

99/120 = 82.5% 

(120) 100% 

2013 (No tutoring) 

5/32 = 15.6% 

27/32 = 84.4% 

(32) 100% 


Table 7: Observed And Expected Counts. Expected Counts In Parentheses. (No Tutoring Only) 



Pass 

Fail 

£ 

2011 (No tutoring) 

31 (32.87) 

176(174.13) 

207 

2012 (No tutoring) 

21 (19.05) 

99 (100.95) 

120 

2013 (No tutoring) 

5 (5.08) 

27 (26.92) 

32 

Sum 

57 

302 

359 


The conditional percentages in Table 6 are very close to each other. So are the observed and expected 
counts in Table 7. Since the X~ statistic measures the difference between the observed and the expected counts it is 
expected to have a low value of the test statistic X 2 . The hypothesis test for this case is defined as follows: 

H 0 (Null Hypothesis): There is NO difference in placement exam passing rates from year to year for the three years 
that data is available (2011, 2012, 2013); therefore, the two variables are independent from each other. 

H a (Alternate Hypothesis): There is a difference in placement exam passing rates from year to year, that is, that 
there is a relationship between the two variables (so that they are not independent). 
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The chi square analysis was also run in the R programming language. The X 2 statistic calculated by R (X 2 = 
0.364), as expected, is very low. For a 3x2 matrix the minimum required value to show association at a 95% 
significance level is X 2 = 5.991. In addition, the p-value calculated by R is 0.8336 which is considerably higher 
than the minimum required p-value (0.05). 

Therefore, the null hypothesis is accepted. 

By accepting the null hypothesis, the implication is that, when there is no tutoring, the passing rates in the 
mathematics placement exams will be low (approximately a 15% passing rate) irrespective of the year the exam is 
taken. 


This result may be generalized into the past and the future (entire population). When looking into the past, 
this Chi-Squared test has established that there is no need to gather and analyze additional data from previous years 
as it will most probably be near 15%. When looking into the future, this Chi-Squared test has established that, if the 
SMTP were to be eliminated, the passing rates in the future would fall once again to the 15% level. This conclusion 
provides a solid and statistically-significant basis to institutionalize the SMTP after the grant ends in 2016. 

CONCLUSIONS AND RECOMMENDATIONS 

After successfully implementing the SMTP for two years, some conclusions can be drawn: 

1. The SMTP has shown to be statistically significant in helping students pass one or more placement exams 
in UT. 

2. Freshmen students showed high compromise and engagement with the SoE during this period. 

3. The program primes the SoE freshmen to additional types of supplementary instruction beyond classic 
lectures. 

4. The improvements made for the second year of the program made it highly successful. 

5. Reception was very favorable amongst freshmen students and their families. 

6. Program promotes the integration between students before they begin their first semester in the UT. 

7. Fear of entering into an advanced course that they will fail was one of the reasons for not taking the 

placement exam. 

Some recommendations for improving this program or any similar program that wants to replicate it are: 

1. Although many students participated in the SMTP (sometimes for more than a week), not everyone took 
the placement exam. Efforts should also focus in getting students to understand the benefit they can reap 
from taking the placement exam. 

2. Implement a similar program during the August/December semester for students that want to enroll in 
semesters that start in January. 

3. Divide students from the start into groups depending on their mathematics entry level. 

4. Maintain close communications with the tutors in order to detect more room for improvement. 
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